MPA 5830 - Module 04

Ani Ruhil

2018-07-23

Agenda

Our goal in this module is to understand some basic ways of visualizing data. We will skip base R commands and instead just work with ggplot2, the most popular visualization package in the R universe.

Remember the basic options…

  • one qualitative/categorical variables: bar-chart
  • one quantitative/continuous variables: histogram/box-plot/area-chart
  • two quantitative/continuous variables: scatter-plot/hex-bin

Two data-sets

I will use two data-sets, the first being this IMDB data-set

The internet movie database, http://imdb.com/, is a website devoted to collecting movie data supplied by studios and fans. It claims to be the biggest movie database on the web and is run by amazon. More about information imdb.com can be found online, http://imdb.com/help/show_ leaf?about, including information about the data collection process, http://imdb.com/help/show_leaf?infosource.

##  [1] "title"       "year"        "length"      "budget"      "rating"     
##  [6] "votes"       "r1"          "r2"          "r3"          "r4"         
## [11] "r5"          "r6"          "r7"          "r8"          "r9"         
## [16] "r10"         "mpaa"        "Action"      "Animation"   "Comedy"     
## [21] "Drama"       "Documentary" "Romance"     "Short"

A data frame with 28819 rows and 24 variables

  • title. Title of the movie.
  • year. Year of release.
  • budget. Total budget (if known) in US dollars
  • length. Length in minutes.
  • rating. Average IMDB user rating.
  • votes. Number of IMDB users who rated this movie.
  • r1-10. Multiplying by ten gives percentile (to nearest 10%) of users who rated this movie a 1.
  • mpaa. MPAA rating (missing for a lot of movies)
  • action, animation, comedy, drama, documentary, romance, short. Binary variables representing if movie was classified as belonging to that genre.

The second data-set is the Star Wars dataset, a tibble with 87 rows and 13 variables:

##  [1] "name"       "height"     "mass"       "hair_color" "skin_color"
##  [6] "eye_color"  "birth_year" "gender"     "homeworld"  "species"   
## [11] "films"      "vehicles"   "starships"
  • name: Name of the character
  • height: Height (cm)
  • mass: Weight (kg)
  • hair_color,skin_color,eye_color: Hair, skin, and eye colors
  • birth_year: Year born (BBY = Before Battle of Yavin)
  • gender: male, female, hermaphrodite, or none.
  • homeworld: Name of homeworld
  • species: Name of species
  • films: List of films the character appeared in
  • vehicles: List of vehicles the character has piloted
  • starships: List of starships the character has piloted

a tibble you say?

dataframes vs. tibbles

R’s default is to store a dataframe, as shown below with a small example and there is a tendency to convert characters into factors, change column names, etc.

## 'data.frame':    3 obs. of  2 variables:
##  $ Some.Letters: Factor w/ 3 levels "A","B","C": 1 2 3
##  $ Some.Numbers: num  1 2 3
##   Some.Letters Some.Numbers
## 1            A            1
## 2            B            2
## 3            C            3

tibbles is the brainchild of the team behind a bundle of packages (and RStudio) called the tidyverse that drops R’s bad habits

## Observations: 3
## Variables: 2
## $ `Some Letters` <chr> "A", "B", "C"
## $ `Some Numbers` <dbl> 1, 2, 3
## # A tibble: 3 x 2
##   `Some Letters` `Some Numbers`
##   <chr>                   <dbl>
## 1 A                           1
## 2 B                           2
## 3 C                           3

There are other advantages to tibbles that we will encounter at a later stage.

ggplot2 and the grammar of graphics

<img src = “../images/grammarofgraphics.png”, width = 400px>

qplot will generate a quick plot but ggplot2 is the way to go so we build with it. Read the relevant chapter on the grammar of graphics from the link in the syllabus or then watch this video.

Nothing results since we have not specified how we want the variable(s) to be mapped to the coordinate system… what variable should go on what axis?

Now we are getting somewhere. We see the canvas with the specific eye colors on the x-axis but nothing else has been drawn since we have not specified the geometry … do you want a bar-chart? histogram? dot-plot? line-chart? what??

With a categorical variable the bar-chart would be appropriate and so we ask for a geom_bar()

Other aesthetics can be added, such as group, color, fill, size, alpha , axis labels, plot title/subtitle etc.

There are two commands for adding a color scheme – color or colour versus fill

Note what colour = generated for us, and how this differs from fill = (see below).

Of course, it would be good to have the colors match the eye-color so let us do that next.

R Colors used from are this source but see also this source. Colors can be customized by generating your own palettes via the Color Brewer here. But don’t get carried away: Remember to read the materials on choosing colors wisely, particularly the point about qualitative palettes, divergent palettes, and then palettes that work well even with colorblind audiences.

I’ll switch to a different variable and show you how to use prebuilt color palettes.

Now add labels, a title, subtitle, and use a color scheme, preferably with scale_fill_brewer and then at least one with scale_fill_manual

Study the commands carefully and note that

  • scale_fill_brewer is being used in the first plot, calling on built-in color palettes. You can review them here
  • scale_fill_manual is being used in the second plot and a specific palette is being invoked from the wesanderson package

Color palettes will come into play far more later on in this course.

More with bar-charts

Notice that we switched the aes() piece of the code but that made no difference; this is important to bear in mind because it will come in handy down the road when we need to build some advanced visualizations.

The plot is sub-optimal since MPAA ratings are missing for a lot of movies and should be eliminated from the plot via subset(mpa != "") or by running dplyr’s filter() to create another data-set.

##  chr [1:58788] "" "" "" "" "" "" "R" "" "" "" "" "" "" "" "PG-13" ...

The order of the bars is fortuitous in that it goes from the smallest frequency to the highest frequency, drawing the reader’s eye. I said fortuitous because the default is to order the bars in an ascending alphabetic/alphanumeric order if the variable is a character. See below for an example.

Later on we’ll learn how to order the bars with ascending/descending frequencies or by some other logic.

What about plotting the relative frequencies on the y-axis rather than the frequencies?

Note the addition of

  • y = (..count..)/sum(..count..)
  • scale_y_continuous(labels = scales::percent)

We could also add a second or even a third/fourth categorical variable. Let us see this with our hsb2 data-set. we can start by reading in the data file.

This is not very useful since the viewer has to estimate the relative sizes of the two colors within any given bar. That can be fixed with position = "dodge", juxtaposing the bars for the groups as a result, and the end product is much better.

This is fine if you want to know what percent of the 200 students are low SES males, low SES females, etc. What if you wanted to calculate percentages within each sex?

What about within each ses?

Histograms

If you’ve forgotten what these are, see histogram, or then Yau’s piece here and here. There is a short video available as well.

Let us load the hsb2 data we had downloaded, processed (adding value labels to categorical variables) and saved in our data folder in the last module.

For histograms in ggplot2, geom_histogram() does the trick but note that the default number of bins is not very useful and can be tweaked, along with other embellishments.

We could set bins = 5 and we could also experiment with binwidth =

If we wanted to break out the histogram by one or more categorical variables, we could do so quite easily:

Or better yet,

since now the distributions are stacked above each, easing comparisons.

One useful design element with breakouts is placing in relief the consolidated data (i.e., the distribution without break-outs).

Here it is obvious that the distribution of readings scores of any one sex are similar to the overall distribution so perhaps the groups are not really that different in terms of reading scores

For breakouts with two categorical variables we could do

Note that ~ female + schtyp renders the panels for the first category of female by all categories of schtyp and then repeats for the other category of female.

Note that schtyp ~ female renders the panels for the first category of schtype for all categories of female and then repeats for the other category of schtyp

… which is the same as …

In general, do not forget to set the y limit to start at 0 or then make a note in the plot for readers so they don’t assume it is at 0 when in fact it has been truncated for ease of data presentation. If this misstates the pattern in the data, do not do it or then, again, annotate the plot to that effect so nobody is misled. Bar-charts will have 0 as the minimum y-limit but not so for histograms and some other plots involving continuous variables.

Ridge-plots

These were all the rage in the summer of 2017, and named joy plots but the unfortunate connection with the source of the plots led the name to be revised to ridge-plots. If you are curious, see why not joy?. You need to have installed the ggridges package but other than that, they are easy to craft.

Here is another one, mapping the distribution of hemoglobin in four populations (the US being the reference group) as part of a study looking at the impact of altitude on hemoglobin concentration (courtesy Whitlock and Schluter).

As should be evident, they are visually appealing when comparing a large number of groups on a single continuous variable and using simple facet-wrap or other options would be unfeasible.

Line-charts

These are useful for time-series data since they map trends over time.

They can look very plain and aesthetically unappealing unless you dress them up. See the one below and then the one that follows.

Here is the more aesthetically pleasing version built using plotly

Scatter-plots

These are great with two continuous variables, and work well to highlight the nature and strength of a relationship between the two variables …. what happens to y as x increases? s

We could lean on ggplot2 and highlight the different ses groups, to see if there is any difference.

This is not very helpful so why not breakout ses for ease of interpretation?

And then of course we could make it interactive with plotly …

Count plots

count plots show the frequency of given pairs of values by varying sizes of the points. The more the frequency of a pair, the greater the size of these points. Useful but somehow I don’t end up using them much.

The second example relies on our Boston Marathon data, looking at finishing times of men and women, respectively. We could have tried to put both groups in the same plot but that ends up obscuring things more than revealing anything.

Hex-bins

Scatter-plots and count plots are not helpful when data points overlap. This is where hex-bins come in handy. In brief, they carve up the plotting grid into hexagons of equal size, count how many \(x,y\) pairs fall in each hexagon, and use a color scheme (like a heatmap) to show where hexagons have more data versus less.

We could add a third variable, diamond color, for example.

Some reminders about ggplot2 rules & other resources

  • basic structure: ggplot(data, aes()) + geom_(aes()) + ...
  • aes() will take x =, y =, fill =, color =, group =, size =, radius =, size = and more
  • each geom has its own components
  • plenty of themes available; see for e.g., ggthemes here
  • don’t forget to stay in touch with development of ggplot2 extensions
  • of course, the plotly site and Carson Sievert’s plotly book
  • keep ggplot2 cheatsheet handy
  • join stackoverflow but if you ask a question, post with a MWE (minimum working example). This is crucial otherwise be prepared to have your head bitten off or at best have nobody respond to your question!

Some resources to bear in mind:

And finally, my suggestion of how to go about building your visualizations:

  • 🔁 start with pencil and paper, sketch prototypes of desired visualization(s)
  • 😄 graphics are relatively easy to generate with base R & with ggplot2
  • 👏 common-sense: number & type of variable(s) guide plotting
  • 🎇 stay color conscious: sensible colors & sensitive to color blindness
  • 🔰 experiment, experiment, experiment until you are happy
  • use the 🆓 learning resources available online
  • 📒 if you learn something new in R, write it down

Practice tasks

Lord of the Rings trilogy data

Use the Lord of the Rings data emailed to you to answer the following questions. Note that these data are from jennybc and represent the number of words spoken by characters in the LOTR trilogy. Some other, pretty amazing visualizations can be seen here, the work of Nadieh Bremer. You are merely looking at how many times a particular race or character appears on screen with a dialogue of at least one word.

  1. Generate an appropriate chart that shows the distribution of Race

  2. Now break this distribution out by Film to see how Race is distributed across Film.

  3. Now generate an appropriate chart to show the distribution of Character by film. Use coord_flip() to flip the coordinates so that the characters show up on the y-axis.

  4. Now use facet_wrap() to generate the three-panel layout, one panel per film.

  5. Use an appropriate chart to plot the distribution of the number of words spoken overall

  6. Now break up this chart by movie.

  7. What if you did it by Race? Which race seems to speak the most?

Water levels in the Great Lakes

Download the monthly Great Lakes water level dataset SPSS format from here and Excel format from here. Note that water level is in meters.

Use the following command to read in the excel file:

Now use an appropriate chart to show the water level for Lake Superior.

County Health Rankings

Download the 2017 County Health Rankings data SPSS format from here, Excel format from here and the accompanying codebook.

Construct appropriate plots that shows the relationship between the following pairs of variables

  1. Adult obesity and High school graduation

  2. Children in poverty and High school graduation

  3. Preventable hospital stays and Unemployment rate

Unemployment Rates

Use the unemployment data given to you and construct appropriate plots that show the distribution of unemployment rates across years for each of the four educational attainment groups.

See if (a) you can pretty the labels for educational attainment (for eg., anything like BA_or_more is ugly), and (b) order the groups such that educational attainment shows up in the plot legend as the ordinal variable that it is. Then regenerate the plot.